Multimodal Emotion Recognition From EEG Signals and Facial Expressions

Emotion recognition has attracted a lot of attention in recent years. It is widely used health care, teaching, human-computer interaction, and other fields. Various human emotional features are often to recognize different emotions. Currently, there more research on multimodal emotion based the fusion multiple features. This paper proposes deep learning model electroencephalogram (EEG) signals facial expressions achieve an excellent classification effect. First, pre-trained convolution neural network (CNN) extract from expressions. Next, mechanism introduced critical frame Then, we apply CNNs spatial original EEG signals, which use local kernel global learn left right hemispheres channels all channels. After feature-level fusion, expression fed into classifier for recognition. conducted experiments DEAP MAHNOB-HCI datasets evaluate performance proposed model. The accuracy valence dimension 96.63%, arousal 97.15% dataset, while 96.69% 96.26% dataset. experimental results show that can effectively carry out